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r— i" Abstract 

H 

C/3 ■ Markov chain models are used in various fields, such behavioral sciences or econo- 

metrics. Although the goodness of fit of the model is usually assessed by large sample 
approximation, it is desirable to use conditional tests if the sample size is not large. 
We study Markov bases for performing conditional tests of the toric homogeneous 
Markov chain model, which is the envelope exponential family for the usual homo- 
| geneous Markov chain model. We give a complete description of a Markov basis 

J> ' for the following cases: i) two-state, arbitrary length, ii) arbitrary finite state space 

and length of three. The general case remains to be a conjecture. We also present 
\ a numerical example of conditional tests based on our Markov basis. 

CO 

: 1 Introduction 

o 

Consider a Markov chain Xt, t — 1, . . . , T(> 3), over a finite state space S = {1, . . . , S} 
(S > 2). Let 7Tj, i G S, denote the initial distribution of X\ and let pfj = P(X t+ i = j \ 
X t = i) denote the transition probability from time t to t + 1. We want to test the null 



X 

H 

hypothesis of homogeneity 



H : p?)= Pij , t = l,...,T-l. (1) 



This model is used as a standard model in many fields. See Haccou and Meelis [8( for 
behavioral sciences and Fuhwirth-Schnatter j3] for econometrics. Usually the goodness 
of fit of the model is assessed by large sample approximation (Anderson and Goodman 
[2j], Billingsley 0]). However when the sample size is not large, it is desirable to use 
conditional tests. The Markov basis methodology (Diaconis and Sturmfels 0|) is attractive 
for performing them. 
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In this paper we study Markov bases for performing conditional tests of the null 
hypothesis of homogeneity. However the homogeneous Markov chain model (PQ) is a curved 
exponential family due to the constraints J2j<=sPij = 1 5 Vi G «S, and we cannot directly 
apply conditional tests to H in (JTJ. Instead, by the Markov basis methodology we 
can test a larger null hypothesis H Q that X t ,t = 1, . . . ,T, are observations from a toric 
homogeneous Markov chain (cf. Section 1.4 of Pachter and Sturmfels [121]). which is the 



envelope exponential family (cf. Kuchler and S0rensen lOj] and Kuchler and S0rensen 



111 . Chapter 7]) of the homogeneous Markov chain model ([!]). In Section I2TT1 we discuss 



interpretations of the toric homogeneous Markov chain model and its difference from the 
usual homogeneous Markov chain model. 

In the following for notational simplicity we write THMC for "toric homogeneous 
Markov chain". Note that in Section 1.4 of Pachter and Sturmfels 12] they consider a 
model without parameters for the initial distribution, whereas our THMC model contains 
parameters for the initial distribution. As an alternative hypothesis to THMC model we 
can take non- homogeneous Markov chain model, which is an (affine or full) exponential 
family model which includes THMC model as a submodel. 

In this paper we derive complete description of Markov bases for THMC model for the 
case of S = 2 (arbitrary T) and the case of T = 3 (arbitrary S). For other combinations 
of (S, T) with small S and T, we can use a computer algebra package 4ti2 ([![). Complete 
description of Markov bases for general S and T seems to be a difficult problem at present. 

The organization of this paper is as follows. In Section [2] we first set up our model 
and then give some preliminary results. We derive a Markov basis for the case of S = 2 
in Section [3] and for T = 3 in Section HI In Section [5] we present an application of our 
Markov basis to a data set. In Section [6] we give some discussions on Markov bases for 
general S and T. 



2 Toric homogeneous Markov chain model 
2.1 THMC model and its interpretation 

Let to = (si, . . . , St) G S t denote an observed path of a Markov chain. Each path 
is considered as one frequency of an \S\ T contingency table. The usual homogeneous 
Markov chain model JT]) specifies the probability of the path u as 

H : p(u) = n Sl p SlS2 . . .p ST _ lST , (2) 

where the normalization is X^i&s 7r i = 1 an d ^ZjesPij — 1> Vz G S. In THMC model, the 
normalization is only assumed for the total probability. In order to avoid confusion, let 
7i, i G <S, and /3y, i,j G S, be free nonnegative parameters. We specify THMC model by 

H ■ p(w) = cy Sl p siS2 . . . /3 ST „ lST , (3) 

where c is the overall normalizing constant. THMC model ([3]) is the envelope exponential 
family of the homogeneous Markov chain model (j2j), i.e., the linear hull of logarithms of 
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the probabilities in ([2]) coincides with the set of logarithms of probabilities in fl3]). This 
simply follows from the fact that the linear hull of elementwise logarithms of probability 

vector 

{(logpi, . . . ,logp s ) | pi > 0,i E S, = 1} 

ies 

is the whole R s . The THMC model Hq contains the usual homogeneous Markov chain 
model Ho and the difference in degrees of freedom is S — 1. 



As discussed in Kiichler and S0rensen [10j an envelope exponential family is often 



difficult to interpret. For THMC model we give the following interpretation. Write 

Pij O^iPij i ^ Pij 1 • 

Then is written as 

Compared to d2J), «j specifies a magnifying ratio of probability of a path visiting the state 
i. We can interpret ctj as an internal growth rate or "birth rate" of state i, which is not 
affected by immigration or emigration. 

As an alternative hypothesis we can take the non-homogeneous Markov chain model 

H 1 : p(u) = n Sl p^ S2 ...£ T -J T . (4) 

By reparametrization this model can be simply written as the linearly ordered conditional 
independence model: 

This is a decomposable model of a T-way contingency table, with its independence graph 
given by the following linear tree: 

1 2 ... T — 1 T 



This model is a full exponential family model containing THMC model. 

In this paper we consider testing H$ against H\ via Markov basis approach. Although 
by this procedure we are not directly testing H , it should be noted that we can reject H 
if H is rejected. When H is accepted, we further need to test H against H . However 
at present there is no general procedure for finite sample exact tests of curved exponential 
family. 



2.2 Sufficient statistic and moves for THMC model 

Suppose that we observe iV paths ui, . . . ,con from a homogeneous Markov chain (Q. We 
write W = {coi, . . . ,oj n }. Here multiple paths are allowed in W and hence W is a multiset. 
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Let x(u) denote the frequency of the path oj in W. Then the data set is summarized in 
a T-way contingency table x = {x(u),oj 6 S T }. 

Let x\a denote the number of transitions from s t — i to s t+ i — j in W and let x\ 
denote the frequency of the state s t = i in W. In particular x\ is the frequency of the 
initial state s\ = i. Let 



T-l 



E 

t=i 

denote the total number of transitions from i to j in W. Under H the joint probability 
of x is written as 

P ^ = ff II (W-i-a • • -Pst-ist)^ 



iV! 



II 1L 

ses i,jes 



X. ■ 



Therefore the sufficient statistic for H is given by the initial frequencies and the frequen- 
cies of transitions 

b = b ( x ) = { x l ieS } u {x±,i,jeS}. (5) 

In fact this sufficient statistic for Hq is minimal sufficient and it is also the sufficient 
statistic for THMC model. This simply reflects the fact that THMC model is the envelope 
exponential family of the homogeneous Markov chain model. Since THMC model ([3]) is 
an exponential family with integral sufficient statistic, we can perform the conditional 
test of Hq by the Markov basis methodology. Therefore our goal in this paper is to obtain 
Markov bases for H with various S and T. 

If we order paths of S T appropriately and write x as a column vector according to the 
order, b = b(x) in (jSJ) is written in a matrix form 

b = Ax, 

where A is an S(S + 1) x S T matrix consisting of non-negative integers. The set of all 
contingency tables sharing b is called a fiber and denoted by Tb = {x G Z> | Ax = b}, 
where Z> = {0, 1, . . . }. A move z for THMC model is an integer array satisfying Az = 0. 
Then z is expressed by a difference of two contingency tables x and y in the same fiber: 

z = x — y, z(u) = x(co) — y(u), u E S T . 

We write z\ = x\ — y\ and z\- = x\^ — y\y Note that ^ i£S z\ = for all t, because 
iV = Ylies x t = SiG5 v\ i s t ne number of paths. Also since x and y is in the same fiber 

J2t=i 4 = for a11 eS - 

Example 1. For illustration we consider the case of S = 2 and T = 4. The configuration 
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A is written as 



1111 1112 1121 1122 1211 1212 1221 1222 2111 2112 2121 2122 2211 2212 2221 2222 
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(6) 



The first two rows of A correspond to x\ and x\. The other rows correspond to x\ x , 
%i 2 , xti> x 22 respectively. The columns of A are indexed by the paths u G {1,2} 4 in the 



lexicographic order. Note that compared to a similar configuration in Section 1.4-1 of 11 2 1. 
A in (0|) has additional two rows corresponding to initial frequencies. 

An interesting feature of the above A is that it contains identical columns. The columns 
1121 and 1211 are identical. This means that the difference of the two paths z = {z(u) | 
coeS T } 

( 1, if u = 1121, 
z (cj) = ) -1, ifuj = 1211, 
[ 0, otherwise 

forms a degree one move. We depict the degree one move as 



s \* 1 2 3 4 

1 w'7 , (7) 

2 ■ V • 



where a solid line from (i, t) to (j, t + 1) represents zjj = 1 and a dotted line from (i, t) to 
(j,t+ 1) represents zjj = —1. We note that the columns 2122 and 2212 are also identical. 
We could remove redundant columns from A and only leave a representative column from 
the set of identical columns. However it seems better to leave identical columns in view of 
the symmetry in A and we will work with the above form of A. 

In the following we often use the graph representation of moves as ([7]). We call such 
a graph a move graph. A node of a move graph is a pair (i, t) of state i and time t 
and an edge from (i,t) to (j, t + 1) represents the value of z\y If \z\j\ = 0, there is no 
corresponding edge in the graph. If > 2, we write the value of |z*-| beside the edge. 
For example, the move for S = 2 and T = 4 such that z\ x = z\ 2 — — 1, z\ 2 — —2 and 
z\ 2 = z 22 = z\ x — zf 2 = 1 is represented by 



We note that a move graph does not have a one-to-one correspondence to a move and 
more than one move have the same move graph in general. 
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For notational simplicity we denote the edge from (i,t) to (j,t) as t : ij. Given 
a contingency table x, x\ is the number of paths passing through the node (i,t) and 
x\a is the number of paths passing through the edge t : ij. Consider a partial path 
(s t , s t+ i, . . . ,s t r) starting at time t and ending at time t! > t. We write this partial path 
as t : s t s t+ i . . . s t >. x\ t , denotes the frequency of the partial path t : s t s t+ i . . . s t > in 
W. For a particular path ui G S T we say that co passes through t : s t s t+ i . . . s# if cu is of 
the form (*,...,*, s t , s t+ i, . . . , s t r, *,..., *) where * is arbitrary. 

Let z = x — y. If z\ > we say that x dominates y at the node (i,t). Similarly if 
zjj > we say that x dominates y in the edge t:ij. Consider a partial path t:s t ...s t >. If 
all of z\ uSt+v z^; ua , . . . , 2*' 7^,8 , are positive, we say that « dominates y in the partial 
path t:s t ...s t >. 

Given the signs of some zl-'s, we depict the pattern of signs as an undirected graph 
with thick edges. Positive edges are depicted by thick solid lines and negative edges by 
thick dotted line. In the case of S = 2 and T = 4, when z\ x , z\ 2 and z\ 2 are positive and 
z\ 2 , zf 1} z\ 2 and z\ 2 are negative, the corresponding graph is depicted by 

s\t 1 2 3 4 




regardless of the signs of ^2? ^21 5 ^li an( i ^11 where no edge is depicted. We call such 
graphs edge-sign pattern graphs. We note that edges of an edge-sign pattern graph have 
only the information on the signs of z\j and do not have the information on the value of 
zjj like a move graph. 

For considering our Markov basis it is important to note that THMC model possesses 
the symmetry with respect to time reversal. The following fact is well known, as discussed 
in Section 2 of Billingsley 

Lemma 1. For all x in the same fiber J-5 of THMC model, the terminal frequencies xj, 
j G S, are common. 

In terms of moves, the above lemma implies that zj = for all i G S. Since z\ = 0, 
we have 

T-l 

5>' = °- (9) 

t=2 

Lemma [1] shows that given b, the paths of the original homogeneous Markov chain 
and the paths of the time-reversed homogeneous Markov chain has the same conditional 
distribution. This symmetry with respect to time reversal is important in considering a 
Markov basis for the present problem. 
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2.3 Linearly ordered conditional independence model and cross- 
ing path swapping 

Let x and y in the same fiber be given. We call x and y edge-wise equivalent if zjj = 
for all t and for all Edge-wise equivalence does not mean that x and y are identical 
as a multiset of N paths. However consider a non- homogeneous Markov chain i^i in (j4]), 
which is a linearly ordered conditional independence model of T-way contingency tables. 
Then a;*-, 1 < t < T — 1 and i,j G S, constitute the sufficient statistic for the model. 
Therefore moves for Hi is expressed by the difference of two edge-wise equivalent paths 
and their move graphs have no edges. 

We note that Hi is a decomposable model for contingency tables. By Dobra \^ there 
exists a Markov basis for this model consisting of square-free degree two moves. These 
square-free degree two moves are related to the idea of swapping of two paths meeting 
(or crossing) at a node. Let u) = (si, . . . , St) and u' = (s[, . . . , s' T ) be two paths. We say 
that these two paths meet (or cross) at the node (i, t) if i = s t = s' t . If u and u' cross at 
the node (i,t), consider the swapping of these two paths like 

{u,u'} = { (si,...,s t -i,i,s t+ i,...,s T ), (s'i,...,s' t _i,i,s' t+ i,...,s' T ) } 

<B- { (s u . . . , s t _ u i, s' t+1 , . . . , s' T ), (s[, . . . , s'^J, s t+u . . . , s T ) } := {u,uj'} (10) 

t 




Then the difference z of (u,u') and (&,&') 




1 if u = u or oj' 
— 1 if u) = u or bj 
otherwise 



forms a move for H\. We call this move a "crossing path swapping". By Dobra [6( we 
have the following proposition. 

Proposition 1. The set of crossing path swappings in / fTTJj) constitutes a Markov basis 
for the linearly ordered conditional independence model. 

Therefore once x and y of the same fiber are brought to be edge-wise equivalent, they 
can be further brought to be identical as multisets by crossing path swappings. Therefore 
for constructing a Markov basis for THMC model, we need to find an additional finite set 
of moves, which can make two elements of the same fiber edge- wise equivalent. 

Crossing path swapping has the following implication. Suppose that x dominates y in 
the partial path t:s t . . . s t >. Then clearly there exists a path w in a; passing through the 
edge t : s t s t+ i. Similarly there exists a path ui' in x passing through the edge t+ 1 : s i+ iS 4+2 . 
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We can then make the crossing path swapping of these two paths and make ui go through 
the partial path t : s t s t+ iS t+ 2- Repeating this procedure, by a sequence of crossing path 
swappings of paths of a?, we can assume that x contains a path u which goes through the 
partial path t:s t ■ ■ ■ s t i. We state this as a lemma. 

Lemma 2. Let x and y be two elements of the same fiber. If x dominates y in the partial 
path t:Sf. . .Sf, then by crossing path swappings of paths in x, we can transform x to x, 
such that there exits a path in x going through the partial path t:s t . . . s# ■ 

Note that during the sequence of crossing path swappings, the values of {zjA stay the 
same. By Lemma EJ we can identify all cc's with the same edge frequencies, whenever we 
are concerned about decreasing Ylti j \ z lj\ l° r our distance reduction argument in Sections 



2.4 Some properties of moves for THMC model 

In this subsection we present an important class of moves for THMC model. Consider two 
different states i% ^ i 2 at time t and two different states j\ 7^ j 2 at time t + 1. Let t:ixj% 
and t : ^2j2 be two edges joining these states. Now consider t' > t and two edges t' : i\j 2 
and t' :i 2 ji, which swaps the transitions iiji of time t. Suppose that all of xj^, x\ 2 j 2 , 
x^j 3 , k* 2Ji are positive. Choose (not necessarily distinct) four paths U\, u) 2 , w 3 , a> 4 from x 
passing through t:iiji, t:i 2 j2, t':i\j 2 , t':i 2 j 1 , respectively: 

cox = (sn . . . ,s 1>t -i,ii,ji,s 1>t+ 2, • • • , sit), w 2 = (s 2 i • • • , s 2 ,f-i, h, J2, s 2 ,t+2, $2t) 

w 3 — ( s 31 • • • j s 3,t'-l> h,j2, s 3,t'+2j ■ ■ ■ 1 s 3t)i <^4 = (541 ■ • ■ , S4,t'-lj &2)ilj s 4,t'+2j • - • j S4t)- 

Then we consider swapping the transitions in {t : ixjx , t : i 2 j 2 } and those in {t! : hj 2 , t' : i 2 ji} 
in x as {ui\, u 2 , oj 3 , oj 4 } -b- {£>i, a} 2 , a) 3 , 0)4}, where 

cDi = (Sn . . . ,Si jt _i,ii, j 2 ,S 2 j +2 , ■ ■ ■ ,S2T), ^2 = (-§21 • • • , S 2 ,t-1, «2, jl, Sl,t+2, • • • , Sit), 
^3 — ( s 31 • • • i s 3,t'-l) Hjil) s 4,t'+2, ■ ■ ■ , S4t)j <^4 = (541 • • • j S4,t'-lj &2jj2) s 3,t'+2j • • • j s 3t)- 

The resulting move z = {z(u) \ u G S T } is written as 



We call this move a "2 by 2 swap" . The corresponding move graph is depicted as follows, 



E] and 1J 




1 if oj — oji, . . 
1 if oj — coi, . . 
otherwise. 



w 4 , 
w 4 , 




(11) 
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We denote the set of 2 by 2 swaps sharing the move graph ffTTT) by 

{t:iij 1 ,t:i 2 j 2 } {t':iij 2 , ?:i 2 ji}. (12) 

As noted above, Ui,u} 2 ,Us and are not necessarily distinct. It may be the case that 
Ui = o>3, Ui = cj 4 , u 2 = w 3 or uj 2 = 0J4. In the 2 by 2 swap in ffl2|) and fITT]) . the paths 
are only partially specified. Therefore it corresponds to a set of moves in term of fully 
specified paths. In particular, depending on which paths are the same, a 2 by 2 swap may 
be of degree 2,3, or 4 in terms of the fully specified paths. 

The above 2 by 2 swap can be extended in various ways. First it can be extended to 
a permutation of more states 

{t:iiji, . . . ,t:i m j m } -H- {t'-.iij^, . . . ,t': Wo-( m )}, (13) 

where a is an element of the symmetric group S m . We call this move an "m by m 
permutation". The degree of this move is at most 2m in terms of paths. The degree is 
smaller if o~{l) = I for some I, because swapping t : iiji and t' : iiji = t' : i[j a (i) does not 
change anything. 

Second, it can be extended to swapping of two partial paths 

t : i * ■ ■ • * j -H- t' : i * • • • * j , (t ^ t') , 

k k 

where '*' is arbitrary. For example we can swap t: 112 and t' : 122, (t ^ t'): 

{si . . . Si_ill2s t+3 ...s T ,s[... 4-i1224' +3 . . . s' T } 

O {s 1 . . . s t _!l22st +3 ...s T ,s[... s^ms^ . . . s' T }, (14) 

which can be depicted as 



t f 




(15) 



3 Markov basis for two state space case 

In this section we give an explicit form of a Markov basis for the case of S — 2. The 
length T of the Markov chain is arbitrary. 

Theorem 1. A Markov basis for S = 2, T > 4, consists of the following moves: 

1. Crossing path swappings. 

2. Degree one moves. 



3. 2 by 2 swaps of the following form: 
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4- Moves of the following form: 



This includes the case of a double transitions in the middle: 

For T = 3 a Markov basis consists of the first three types of moves. 

Proof. We use an induction on the length T of the Markov chain combined with the 
distance reduction argument in [14| for the distance \z\ :— J2tij \ z lj\- 



If zjj = 0, Vi, j, then we can ignore the transitions l:ij, i,j G S, in both x and y and 



regard the chain as starting from t = 2. Similarly if zL 1 = 0, Vi,j, then we can regard 



the chain as ending at time t = T — 1. Therefore we can assume that z}a ^ for some 
i,j and zj~ x ^ for some In view of the symmetry of the model and the fact that 
z\ = z\ = 0, we can assume without loss of generality that z\ x > and z\ 2 < 0: 



As the initial step of the induction we assume T — 3. Then by (Q zf = 0, i = 1,2. 
This necessarily implies that the edge-sign pattern of z is of the following form: 

12 3 

By Lemma [2, we can assume that x contains the path 112 and 221. Therefore by adding 
the 2 by 2 swap 

12 3 




to x, we can reduce \z\ by eight. This takes care of the case of T = 3. 

Now consider the case of T > 4. First we explain our strategy. We claim that by 
moves in Theorem [1] we can always achieve one of the following two things: 

1) \z\ is reduced without increasing ^ . \z}j\ + Yliij \ z Jf l \\ 

2 ) Eij 141 + Eij \ z l~ 1 \ is reduced. 

Then either \z\ can be reduced to zero and we are done, or if we can not further decrease 
\z\ then, we can now reduce £V l^l + Eij l^ -1 |> noting that \ z ij\ + J2i,j \ z if 1 \ has 
not increased. By this strategy we can either make \z\ = or employ induction on T. 
Therefore Theorem [1] is implied by the above claim. 

As a preliminary step consider a path of x passing through s = l,t = 2. If this path 
ever moves to s = 2 and then comes back to s = 1 before T — 1, i.e., if this path contains 
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t: 12 and if : 21, t < if < T — 1, then we can apply a degree 1 move of the following form 



1 t'-t + l t if 




and reduce £V ■ without affecting ^ . Therefore in the following we only need 

to consider the case that every path of x passing through s = 1, t = 2 either always stays 
at s = 1 until T — 1 or moves once to s = 2 and then stays at s = 2 thereafter until T — l. 
We now check four possible sign patterns of {zf^ 1 }. 

T-l T T-l T T-l T T -l T 

1) ' "/ 2) \" 3) \ ' 4) 

• • > ' • • ^ •'' • 

1 2 T -IT 



Case 1. 




As shown above, a path in a? passing through the node s = 1, t = 1, 2 has to either stay 
at s = 1 until T — 1 or has to transit once to s = 2 and stay s = 2 thereafter until T — l. 
Consider the latter case. If a transition from s = 1 to s = 2 occurs before T—l, then 
m view of 4f 1 > o, we can assume that the path also contains T — 1 : 21 by Lemma 
Then we can apply a degree one move to x and then £\ . |^| + i s reduced. 

Therefore we only need to consider the case where every path in x passing through the 
node s — 1, t — 1, 2 move to s — 2 at T — 1. By Lemma [2j we can assume that there is 
no transition £ : 12, t — 2, . . . , T — 1 in x and therefore z\ 2 < 0, for £ = 2, . . . , T — 1. Then 
z^2 l > in view of z\ 2 < 0. Interchanging the roles of x and y and reversing the time 
direction we similarly have z\ x < 0. From the fact that z\ = z\ = 0, we have z\ 2 > and 
^22 _1 < 0, i.e. z has a sign pattern 



1 2 T-l T 




Therefore |z| is reduced by eight by a 2 by 2 swap. 

1 2 T- 1 T 
Case 2. *T — • \ * 

Since z\ 2 < 0, z[ 2 l < 0, either there exists 2 < i < T — 1 such that 2*- > 2 or there 
exist 2<t<t'<T — 1 such that zj 2 , zf 2 > 0. In the former case write if = t. By the 
symmetry with respect to time reversal of THMC model, we can assume that a path in 
x passing through the node s = 2,t = T, T — 1 stays at s = 2 until t = 2 or transits once 
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to s = 1 and stays s — 1 until t = 2. By Lemma El we can assume that there exist two 
paths passing through t — 1 : 112 and t' : 122, respectively: 



t t' 



Then by subtracting a move of type 4, we can widen the difference t' — t. By iterating 
this procedure until either t = 1 or t' = T — 1, we can reduce Y\ . 1 2:/ • I + V. . 

1 2 T -IT 



Case 3. 



We can assume 2^ 1 > because we have already covered Case 1. Then there has to be 

): 

1 2 t T-1T 



2 < t <T — 1 such that < 



• • 



Also it can be easily seen that z\ x < (hence > 0) i s reduced to Case 1 by a symmetry 
consideration. Therefore we can assume > 0- Then 

z\ = z\ x + 4 > 0, 

i.e., x dominates y at the node s — l,t — 2. Then z\ < because z\ + z\ = 0. 
Similar consideration shows that z\~ l > and z\~ l < 0. Suppose that a path of y 
passing 1 : 12 arrives at s — 1 at time t. By Lemma [2] we can assume that the path 
stays at s — 1 at time t + 1. Denote the path by w = (12s 3 • • ■ s 4 _ills 4+2 ■ • • St)- Let 
w' = (112s 3 ■ • • s t -ils t +2 ' ' ' St)- Then the difference of w and w' forms a degree one move 
with a sign pattern 



By adding it to x, we can reduce £\ . \ z}-\ by two without affecting £\ . |zy -1 |. Therefore 
we only need to consider the case where there is no path of y passing the node s = 2, t = 2 
and arriving at s = 1 at time t. 

Since z\ > 0, there has to be some t' <t such that > 0. Now assume that t' < T — 2 
and choose the minimum t' with z\ 2 > 0- Then a path in cc passing through (s, t) = (1, 2) 
and t' : 12 has to stay at s = 2 from t' + 1 to T — 1. Recall < -zf -1 = xff 2 + ^2i~ 2 and 
suppose that z\^ 2 > 0. Then we can assume that there exists a path passing through 
T — 2 : 112. Therefore we can apply type 4 move 

t-1 t T-2 T 
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and we can reduce z2ij \ z if I without affecting z2ij \ z ij\- Suppose that z 2 { > 0. Then 
we can assume that there exists a path passing through T — 3 : 2212 and hence we can 
apply a degree one move 

T-2 T 



to reduce Y\. . \zj- 1 \ without affecting Y\ . \zjA. Therefore we only need to consider the 
case of t = t = T — 2. Since z 1 _ > and 2; n _ < 0, we have z 2 { > 0. In the same way 
we also have zf 2 ~ 2 < 0. Hence z has a sign pattern 



T-2T-1 




By symmetry we can easily see that z has a sign pattern 

2 3 T-2T-1 

Z X 

Then we can decrease |z| by a 2 by 2 swap. 

1 2 T -l T 
Case 4. *T — • * — 7* 

• *• «* • 

By a consideration as in the previous case there exists t such that z\ x < 0: 

1 2 t T-1T 



As in the previous case there should exist (the minimum) t' < t with z\ 2 > 0. By 
symmetry there also exists (the maximum) t" > t + 1 with z\ x > 0. Then we can apply a 
degree 1 move 

1 2 f'-f + l t' t" 




to decrease • | | . This completes the proof of Theorem [TJ 

4 Markov basis for the case of length of three 

In this section we give an explicit form of a Markov basis for the case of T 
number of states S = \S\ is arbitrary 
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□ 



= 3. The 



Let ,imi m < S, be distinct elements of S. Similarly let ji, . . . ,j m , m < S, be 

distinct elements of S. In the m by m permutation of (Tl~3]) let t — 1, t' — 2 and 

cr(l) = m, a(2) = 1, . . . , <r(A;) = m — 1. 

Then the resulting set of m by m permutation is denoted by 

Z (iij • • • , imi Ji? • • • j jm) ■ 

{l:i 1 j 1 ,l:i 2 j 2l ■ ■ ■ A-imjm} {2 :iij TO , 2 ri^, . . . , 2: (16) 

A move graph of a permutation for S = 6 with m = 4, (^1,^2,23,^4) = (1,2,4,5), 
{jii J2, J3, J4) = (1,3,5,6) is depicted in Figured! Now we state the following theorem on 



s\t 1 2 3 




Figure 1: A typical permutation 
a Markov basis for the case of T = 3. 

Theorem 2. ^4 Markov basis for THMC model with T = 3 is given by the set of crossing 
path swappings in [Tty) and the set ofmxm permutations Z(i\, . . . , i m ; j%, . . . , j m ) in (ESP, 
where m = 2, . . . , S, i\, . . . , i m are distinct, and ji, . . . ,j m are distinct. 

For proving Theorem [2] we need more detailed consideration of these moves. 
Let 

X= {ii,...,i m }, J= {ji,...,j m }. 
For each j G X fl J7", there exist unique indices i(j), G X, k(j), k'(j) G ,7, such that 

l:«(j)j £ {l:«iii, l:z 2 j2, • • • , 1: Wm}, 
Z'-jk'(j) G {2:iiji,2:i 2 j2, • • • ,2:z m j m }, 
l:«'C7')j e V-'iljm, l:«2ji, • • • , 1: Wm-i}, 
Z'-jk(j) £ {2:i]J m ,2:i2ji,...,2:i m j m _i}. 
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For example consider j — 1 in Figure [TJ Then = 116 is a solid path and 

i'(l)lk'(l) = 211 is a dotted path. Next, for each j G X \ J, there exist unique indices 
k(j),k'(j) such that 

2:jk(j) € {2:iij rm 2:i 2 j 1 , . . . , 2 :z m j m _i}, 2:jk'(j) G {2:«ij 1 ,2:i 2 j2, • • • ,2:i m j m }. 

For j = 2 in Figured! A;(j) = 1, fc'(j) = 3, 2:21 is a solid edge and 2:23 is a dotted edge. 
Finally for each j G J \ X there exist unique indices such that 

e {' i --hjiA-hj2, ■ ■ ■ A-imjm}, e {I'-hjmA'-hji, ■ ■ ■ , 1: Wm-l}- 

For j = 3 in Figure [H = 2, =4, 1 : 23 is a solid edge and 1 : 43 is a dotted edge. 
Define the set of paths W\ and W 2 as follows, 

W 1 = {i(j)jk(j),j elnj}u { Sj jk(j),j el\J}u {i(j)js j} j ej\l} 
W 2 = {if{j)jk'(j),j g x n x} u { Sj jk'(j),j el\J}u {i'(j)jh,3 

where Sj,j G X\J and Sj,j G J\X are arbitrary states. Then we consider a set of moves 

Z(h, ■ ■ .,im'Jl> ■■■Jm) = I ■ -ihn'Ju ■ ■ ■ Jm) |wG 5 T }, 

1 if w G Wi 

g(a; | ii, . . . ,i TO ; ji, . . . ,j m ) = { -1 if to G W 2 (17) 

otherwise 

Z(ii, . . . , z m ; ji, . . . , j m ) is easily shown to be an m by m permutation. Now we have the 
following proposition, which implies Theorem [2j 

Proposition 2. A Markov basis for THMC model for T = 3 is given by the set of 
crossing path swappings in (T77J) and moves of Z(ii, . . . ,i m ;ji, . . . ,j m ) of the form (Tty, 
where m = 2, . . . , S, ii, . . . ,i m are distinct, and ji, . . . ,j m o^re distinct. 

Proof. For T = 3, implies that node frequencies are common for two elements x and 
y of the same fiber. Therefore the edge-sign pattern graph G has the following property 
(PI): 

If zjj > (< 0), there exists i' such that z]^ < (> 0). 
If zfj > (< 0), there exists f such that zh < (> 0). 

Clearly similar property holds for initial nodes at t = 1 and terminal nodes at t — 3. Also 
for T = 3 we have zjj + zfj = and therefore zjj > (< 0) if and only if zfj < (> 0). 
This means that G for t = 2, 3 is a copy of G for t — 1, 2 with sings of edges reversed. 

Suppose that x and y are not edge- wise equivalent. Then by the above properties G 
has some positive edge 1 : ?i Ji ■ We follow the edge from t = 1 to t = 2. Then by (PI), we 
can follow a negative edge 1 : i 2 ji back to t — 1. Then we can again follow a positive edge 
1 : to t = 2, etc. By continuing this, the same node is eventually visited twice and a 
loop is formed. By considering the shortest loop, it can be easily shown that there exists 
the following simple loop of G for t = 1,2: 
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There are distinct indices ii, . . . ,i m and distinct indices ji, . . . ,j m such that 
1 : njt, 1 : i 2 j 2 , • • • , 1 : i m j m are positive edges of G and 1 : iij m , 1 : i 2 j u . . . 1 : 
imjm-i are negative edges of G. 

Now we employ the distance reduction argument of Takemura and Aoki [14]. As 
above let X = . . . ,i m } and J = {ji, . . . ,j m }- By Lemma [2] we can assume that 
x(i(j)jk(j)) > for j G X fl J . For j G X \ ,7 we can find a path uj = Sjjk(j) with 
x(oo) > 0. Similarly for j G J7" \ X we can find a path u = such that x{uj) > 0. 

Now we can subtract a move in (flT]) from as. This reduces |z| by 4m. This completes the 
proof of Proposition [2j □ 



5 A numerical example 

In this section we give a numerical example for testing THMC model against the non- 
homogeneous Markov chain model using our Markov basis. 



Table 1: Marijuana use data 
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Table 1 refers to a longitudinal data from 1977 to 1979 on marijuana use of 120 female 
respondents who were age 14 in 1977 (e.g [l6|). The degrees of dependence are categorized 
into the following three levels, 1. never use; 2. no more than once a month; 3. more than 
once a month. 

We examine the goodness-of-fit of H with (S, T) = (3, 3) by testing H . We use 
Pearson's x 2 as a test statistic. We have x 2 = 11.533. 

We computed the exact distribution of x 2 y ia MCMC with a Markov basis derived in 
TheoremEl We sampled 100,000 tables after 50,000 burn-in steps. Figure |5] represents the 
histogram of the sampling distribution of x 2 and the solid line in the figure is the density 
function of the asymptotic x 2 distribution with degrees of freedom 4. The asymptotic 
p-value and the exact p-value are 0.0212 and 0.0184 and THMC model is rejected. For 
this example the homogeneity of Markov chain model is rejected, since THMC model is 
rejected. However, note that the approximation by x 2 distribution is not very good for 
this data. This is probably due to small frequencies in the data set. 
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Figure 2: Sampling distribution of L via MCMC 

6 Discussion and a conjecture for general case 

From the viewpoint of application, it is clearly desirable to obtain Markov bases for general 
5 and T. By 4ti2 we can obtain a Markov basis for 5 = 3 and T = 5. However currently 
this case seems to be the largest case which can be handled by a software. 

Even if obtaining a Markov basis is difficult, it is of theoretical interest to know some 
properties of Markov bases for general 5 and T. Note that the description of Markov 
basis for 5 = 2 is common for all T > 4. This is very similar to the notion of Markov 



complexity (|9|, [13], [3j]) and we believe that there exists some bound of complexity of 
Markov basis in T for a given 5. 

We have done extensive investigation of edge-sign pattern graphs G for the case 5 = 3 



and T = 4. We will discuss these edge-sign pattern graphs in 15]. It seems that the 
following notion of extended simple loop of the edge-sign pattern graph G is important. 
Here we give only a rough definition of an extended simple loop. An extended simple loop 
is a loop, such that when we are moving towards the future we follow positive edges and 
when we are moving towards the past we follow negative edges of G. Also we require that 
each node is passed at most once. An important example of an extended simple loop for 
5 = 3 and T = 4 is depicted as follows. 



We have used the term "extended" to indicate that the lengths of partial paths may be 
greater than one. If all the edges are of length one (in this case the loop involves only two 
time points t and t + 1), we call it a simple loop. 
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Note that for the case of S = 2, a Markov basis consists of moves, which are sum of 
at most two extended simple loops. For the case of S = 3 and T = 4 we checked that a 
Markov basis consists of moves, which are sums of at most three extended simple loops. 
Therefore our conjecture at this point is that for each S, there exists ks, such that a 
Markov basis consists of moves which are sums of at most k s extended simple loops. A 
stronger conjecture is ks = S. 
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